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10.1.4 Stationary Processes 

We can classify random processes based on many different criteria. One of the important questions that 
we can ask about a random process is whether it is a stationary process. Intuitively, a random process 
t EE ./ } is stationary if its statistical properties do not change by time. For example, for a 
stationary process, X(t) and X(t + A) have the same probability distributions. In particular, we have 

F X{t )(■*) = F X{t+ A)( x )> for all t, t + A G J. 

More generally, for a stationary process, the joint distribution ofX(t 1 ) and X(t~>) is the same as the joint 
distribution oiX(t x + A) and X(t 0 + A). For example, if you have a stationary process X( t), then 

(x(t x ),X(t 2 )) e A j = p[(x{t x + A ),X(t 2 + A)) £ A 

for any set A e R 2 . In sum, a random process is stationary if a time shift does not change its statistical 
properties. Here is a formal definition of stationarity of continuous-time processes. 

A continuous-time random process {X(t), t 6 R } is strict-sense stationary or simply stationary if, for 
all t 2 , t r G R and all A e R, the joint CDF of 

X{t x \X{t 2 ),-,X{t r ) 


is the same as the joint CDF of 


Xf x + X),X{t 2 + X),-,X{t r + X). 

That is, for all real numbers x l5 x 2 , •••, x r , we have 

F X{t x )X{t 2 ) -X(t r )( x l’ x 2’ '"’ x r) = ^X(/j+A)X (/2 + a ) - x (t r + A)( x l’ x 2’ x r>- 

We can provide similar definition for discrete-time processes. 

strict-sense stationary or simply stationary, if for all n x , n 2 , •••, n r 6 Z and all D E Z, the joint CDF of 

X(f x ),X(n 2 ), ■■■,X(n r ) 


is the same as the joint CDF of 


X(n | + D), X(n 2 + D), ■■ •, X(n r + D). 

That is, for all real numbers x l5 x 2 , x r , we have 

F X{n x )X{ ni ) -X(n r )( x l’ x 2’ x r) = F X(n x +D)X(n 2 +D) -X(n r +D)( x l’ x 2’ '"> x r )- 
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Example 10.7 

Consider the discrete-time random process {X(n), n E Z--}, in which the X(n)'s are i.i.d. with CDF 
F X ( n ) (■*) = F{x). Show that this is a (strict-sense) stationary process. 

• Solution 

o Intuitively, since X(n)'s are i.i.d., we expect that as time evolves the probabilistic behavior 
of the process does not change. Therefore, this must be a stationary process. To show this 
rigorously, we can argue as follows. For all real numbers x 1 ,x 2 ,---, x r and all distinct 

integers n x ,n 2 ,---,n r , we have 

F X(n x )X(n 2 ) -X(n r )( x 1> X 2> x r ) 

= F X( n ,) 0 i ) F x(n 2 ) (Xj-Fxt n r ) Or) ( since theX(« ; )'s are independent) 

= F(x x )F(x 2 )-F(x r ) (since F x(t ^{x) = F(x)). 

We also have 

F X(n l +D)X(n 2 +D) -X(n+D)\ x l’ x 2’ 

= F x(n t +D)( x 1 ) F X(n 2 +D ) ( x 2>" F X(n+ D)( x , : ) ( since th eX(n ( + D)'s are independent) 
= F(x x )F(x 2 )--F(x n ) (since F x{n+D) (x) = F(x)). 

In practice, it is desirable if a random process X(t) is stationary. In particular, if a process is stationary, 
then its analysis is usually simpler as the probabilistic properties do not change by time. For example, 
suppose that you need to do forecasting about the future of a process X(t). If you know the process is 
stationary, you can observe the past, which will normally give you a lot of information about how the 
process will behave in the future. 

However, it turns out that many real-life processes are not strict-sense stationary. Even if a process is 
strict-sense stationary, it might be difficult to prove it. Fortunately, it is often enough to show a 
"weaker" form of stationarity than the one defined above. 

Weak-Sense Stationary Processes: 

Here, we define one of the most common forms of stationarity that is widely used in practice. A random 
process is called weak-sense stationary or wide-sense stationary (WSS) if its mean function and its 
correlation function do not change by shifts in time. More precisely, X{t) is WSS if, for all t x , t 2 E R 

and all A e R, 

\.E\X(t{)] = E\X(t 2 )], 

2. E[X(t x )X(t 2 )] = E[X(t ] + A )X{t 2 + A)]. 

Note that the first condition states that the mean function ju^t) is not a function of time, t, thus we can 
write = n x . The second condition states that the correlation function R^t x , t~>) is only a function 
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of r = - t 2 , and not t x and /, individually. Thus, we can write R x (t l , t 0 ) = R%(t\ - /,) = Rj^t). 

Therefore, we can provide the following definition. 

A continuous-time random process {X(t), t E R } is weak-sense stationary or wide-sense stationary 
(WSS) if 

1. Hx(t) = ju x , for all t E R, 

2. Rx(t\, t 2 ) = Rjft i - t 2 ), for all t\, t 2 £ R. 


We can provide a similar definition for discrete-time WSS processes. 

A discrete-time random process {X(n), n E Z } is weak-sense stationary or wide-sense stationary 
(WSS) if 

1. = n x , for all n £ Z, 

2. R x (n |, ?;,) = 77 1 - 77 ,), for all n ^, 77 , £ Z. 


Example 10.8 

Consider the random process |A(/), t E R } defined as 

X(t) = cos(t + U), 

where U ~ Uniformed, 2k). Show that X(t) is a WSS process. 
• Solution 

o We need to check two conditions: 

1. jUj^t) = n x , for all t E R, and 

2. Rj£t x , t 2 ) = R^i ~ t 2 ), for all t x , t , £ R. 
We have 


Mjft) = E\X(t )] 

= E[cos(t + {/)] 

rlx 1 

= J 0 cos(t + u)— du 
= 0, for all t £ R. 


We can also find R x (t l , t 0 ) as follows 
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R£h, t 2 ) = E\x( tl )X(t 2 )\ 

= £[cos(/| + (7)cos(( 9 + (7)] 


= E 


1 


1 


~cos(7| + t 2 + 2(7) + -cosC/j - (-,) 


= E 


1 


■cos^j + to + 2(7) 


1 


-COSC?! - t 2 ) 


r2jr 1 1 

J 0 cos(?j + t 2 + u)— du + ~cos((j - to) 


0 + -cos((j - (-,) 


= -cos((j - ( 9 ), for all (j, ( 2 £ R- 


As we see, both conditions are satisfied, thusX(() is a WSS process. 


Since for WSS random processes, R^t x , to) = R%(t i - to), we usually denote the correlation function 
by R^z), where z = t l ~ t 2 . Thus, for a WSS process, we can write 

R^z) = E[X{t)X(t - r)] = E[X(t + z)X{t)\ (10.1) 

As we will see in Section 10.2, R^(z) is a very useful tool when we do frequency domain analysis. Here, 
we would like to study some properties of R x (z) for WSS signals. Let [X{t), t £ R} be a WSS process 
with correlation function R%(z). Then, we can write 

R/0) = E[X(t) 2 ]. 

The quantity E[X(t) ] is called the expected (average) power in X(t) at time t. For a WSS process, the 
expected power is not a function of time. Since X{t) 2 > 0, we conclude that R^(0) > 0. 

R/0) = E[X(t) 2 ] > 0 


Next, let's consider R^i - z). We have 

R^i - z) = E\X(t)X(t + r)] (by definition (Equation 10.1)) 
= E[X(t + z)X(t)] 

= Rj^(z) (Equation 10.1) 

Thus, we conclude that Rj^z) is an even function. 

Rj{( T ) = Rjd ~ kb for all r e R. 
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Finally, we would like to show that R^t) takes its maximum value at r = 0. That is, X(t) and X(t + r) 
have the highest correlation when r = 0. 

| Rjfa) | < Rj£ 0), for all i£R. 

The proof can be done using the Cauchy-Schwarz inequality: For any two random variables X and Y, 
we have 


\EXY\ < -yjE[X 2 ]E[Y 2 ], 

where equality holds if and only if X = aY for some constant a 6 R. Now, if we choose X = X(t) and 
Y = X(t - r), we obtain 

I Emm - r)] I < ^E[X(t) 2 ]E[X(t-z) 2 ] 

= y[RjmrfP) 

= ^0). 

Therefore, we conclude that | R a (t) | < R^0). Considering these properties, Figure 10.4 shows some 
possible shapes for R a {t). 



Figure 10.4 - Some possible shapes for Rj^z). 


Jointly Wide-Sense Stationary Processes: 

We often work with multiple random processes, so we extend the concept of wide-sense stationarity to 
more than one process. More specifically, we can talk about jointly wide-sense stationary processes. 
Two random processes {X(t), (6 R} and { Y(t), t e R } arc said to be jointly wide-sense stationary if 

1. X(t) and Y(t) are each wide-sense stationary. 

2- R ’ C) = l — ^ 2 )’ 


Example 10.9 
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Let X(t) and Y(t) be two jointly WSS random processes. Consider the random process Z(t) defined as 

Z(t) = X(t) + Y(t). 

Show that Z(t) is WSS. 

• Solution 

o Since X(t) and Y(t) are jointly WSS, we conclude 

1 . = //j^, fi y(t) = // y, 

2. Rj(t x , t 2 ) = R}£t\ - t 2 ), Ry(t x , t 2 ) = Rj(ti ~ t 2 ), 

3 - Rxy*R v ( 2 ) = ~ h )• 

Therefore, we have 

// z (0 = E[X(t) + 7 ( 0 ] 

= E[X(t)]+E[Y(t)] 

= fi x + fly- 


R z (t x , t 2 ) = E[{X(t x ) + Y(t x )){X(t 2 ) + 7(0))] 

= E\X( t] )X(t 2 )] + E[X(t x )Y(t 2 )] + E[Y(t x )X(t x )]E[Y(t x )Y(t 2 )] 
= R)Sf\ ~ ^2) + 1 “ C) + ^Yxih ~ * 2 ) + R^J l “ C)' 


Cyclostationary Processes: 

Some practical random processes have a periodic structure. That is, the statistical properties are 
repeated every T units of time (e.g., every T seconds). In other words, the random variables 

X(t x ),X(t 2 ),-,X(t r ) 

have the same joint CDF as the random variables 

X(t x + T),X(t 2 + T),-,X(t r +T). 

Such random variables are called cyclostationary. For example, consider the random process 
{7(0, t £ R | defined as 

7(0 = Acos(cot), 

where A is a random variable. Here, we have 

/ 2k \ 

XI t + — I = Acos(cot + 2k) 

\ a I 

= Acos(cot) = X(t). 
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2n 

We conclude X(t) is in fact a periodic signal with period T = —. Therefore, the statistical properties of 
X(t) do not change by shifting the time by T units, so X(t) is a cyclostationary random process with 

2 JT 

period T = —. Similarly, we can define wide-sense cyclostationary random processes. 

A continuous-time random process {X(t), t 6 R } is cyclostationary if there exists a positive real 
number T such that, for all ty, t 2 , t r e R, the joint CDF of 

X{t x ),X(t 2 ),-,X(t r ) 

is the same as the joint CDF of 

X{t x + T),X(t 2 + T),-,X(t r + T). 


A continuous-time random process \X{t), t E R } is weak-sense cyclostationary or wide-sense 
cyclostationary if there exists a positive real number T such that 

1. H)M + T) = for all t E R; 

2. R x (t ] + T,t 2 + T) = t 2 ), for all ty, t 2 E R. 


Similarly, you can define cyclostationary discrete-time processes. For example, a discrete-time random 
process j X(n), n E Z | is wide-sense cyclostationary if there exists MEN such that 

1. n^n + M) = ju x , for all n E Z; 

2. Rj^fiy + M,n~> + M) = Rj^ny, 77,), for all w l9 n 2 £ Z. 

Derivatives and Integrals of Random Processes: 

Many real-life systems are described by differential equations. To analyze such systems when 
randomness is involved, we often need to differentiate or integrate the random processes that are present 
in the system. You have seen concepts such as continuity, differentiability, and integrability in calculus 
for deterministic signals (deterministic functions). Flere, we need to extend those concepts to random 
processes. Without going much into mathematical technicalities, here we would like to provide some 
guidelines on how to deal with derivatives and integrals of random processes. 

Let X(t) be a continuous-time random process. We say that X(t) is mean-square continuous at time t if 

lim e\ \X(t+8) ~ X(t) | 2 1 = 0. 

<5^0 L -I 

Note that mean-square continuity does not mean that every possible realization of X(t) is a continuous 
function. It roughly means that the difference X(t + 3) - X(t) is small on average. 


Example 10.10 
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The Poisson process is discussed in detail in Chapter 11. If X(t ) is a Poisson process with intensity A, 
then for all t > s > 0, we have 


X(t) ~X(s) ~ Poisson (A(t - 5 )). 
Show that X(t) is mean-square continuous at any time t > 0. 

• Solution 

o We have 


X(t + S) - X(t) ~ Poisson (AS ). 

Thus, 

lim E[ | X(t + S)~ X(t) | 2 ] = lim AS + (AS) 2 

= 0 . 


It is worth noting that there are jumps in a Poisson process; however, those jumps are not very "dense" 
in time, so the random process is still continuous in the mean-square sense. Figure 10.5 shows a 
possible realization of a Poisson process. 



Figure 10.5 - A possible sample function of a Poisson process. 


We can similarly talk about mean-square differentiability and mean-square integrability. If A(/) is a 
random process, the derivative of X(t), 


d 

Y(t) = ~X(t), 

is also a random process. For nice and smooth processes, the derivative can be obtained in a natural 
way. For example, if you have a random process defined as 

X(t ) = A + Bt + Ct 2 , for all t e [0, 00 ), 

where A, B, and C are random variables, then the derivative ol'X(t) can be written as 
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X (t) = B + 2 Ct, for all t E [0, oo). 

Without trying to go much into mathematical technicalities, here we would like to provide some 
guidelines on how to deal with derivatives and integrals of random processes (assuming some mild 
regularity conditions are satisfied). A key point to note is that differentiation and integration are linear 
operations. This, for example, means that you can often interchange integration and expectation. More 
specifically, you can write 



j 0 E[X(u)\du. 


Similarly, if the derivative ofX(t) is well-defined, we can write 


d 


E 





Example 10.11 

, d 

Consider a random process X(t) and its derivative, X (t) = —X{i). Assuming that the derivatives are 
well-defined, show that 


d 

R xx'^v *?) = h )• 


• Solution 

o We have 


R xx'(t] > C) — E\_X{t^)X (C)] 


= E 




= E 


8 

dt~. 


(x{t x )X(t 2 )} 


St. 


-ElXit^tJ] 


dt R ^ vh)- 


<— previous 
next —> 
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